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Introduction 


This  document  represents  the  final  report  for  the  research  project  entitled  “Nonlinear  Signal 
Classification  using  Dynamical  Systems  Theory”,  ONR  Grant  N00014-99-1-0072,  conducted  by 
Dr.  Jim  Kadtke  at  the  Marine  Physical  Laboratory  in  San  Diego,  California.  This  project  was  a 
follow-on  award  for  a  previous  grant  of  the  same  title,  and  was  intended  to  continue  and  extend 
that  research  agenda.  The  start  and  end  dates  for  this  project  were  1  October  1999  through  30 
September,  2001  respectively.  The  research  objectives  and  major  results  obtained  during  the 
program  are  summarized  below,  in  the  format  specified  by  ONR  for  standard  year-end  reports. 

Long-Term  Research  Objectives 

The  long-term  goal  of  this  research  project  since  its  inception  was  the  development  of  new  signal 
processing  and  data  classification  methods  based  on  ideas  from  the  field  of  nonlinear  dynamics. 
Specifically,  nonlinear  time-domain  modeling  methods  using  closed-form  dynamical  equations 
were  to  be  formulated  in  a  rigorous  way  in  the  framework  of  standard  statistical  signal 
classification  methods.  The  aim  was  to  provide  new  algorithms  and  automated  tools  which  could 
enhance  existing  Navy  systems  for  Sonar  detection  and  classification,  and  to  eventually 
transition  them  to  the  fleet.  Also,  this  project  supported  with  theoretical  research  a  second  ONR 
research  program  under  Dr.  John  Tague,  which  focused  on  the  practical  design  and  validation  of 
dynamical  classifiers. 

During  the  initial  grant  period,  significant  results  were  obtained  on  understanding  the  properties 
of  differential  equations  as  signal  classifiers,  both  from  an  analytic  and  a  computational 
viewpoint.  Within  this  last  two-year  program  period,  the  focus  shifted  towards  developing 
methods  for  calculating  optimal  dynamical  signal  models  for  given  data  sets,  and  also  for 
adapting  numerical  methods  to  improve  overall  classification  performance.  These  results  are 
summarized  in  the  remaining  part  of  this  report. 

S&T  Objectives 

Under  the  original  research  program,  we  sought  to  develop  new  time-domain  signal  models 
based  on  closed-form  nonlinear  dynamical  systems,  and  we  developed  techniques  based  on 
delay-differential  equations.  We  also  developed  numerical  processing  schemes  to  utilize  these 
models  along  the  lines  of  standard  signal  processing  algorithms,  and  to  assess  their  usefulness  by 
generating  standard  performance  measures  such  as  ROC  curves.  We  then  sought  to  characterize 
their  performance  on  real-world  data  sets,  such  as  Sonar  and  bio-sonar  data  obtained  from  US 
Navy  and  other  sources. 

In  the  most  recent  two-year  program  period,  we  attempted  to  refine  these  methods  by  developing 
schemes  for  deriving  “optimal”  signal  models  which  could  provide  enhanced  performance  over 
the  generic  models  which  we  had  previously  utilized.  We  also  developed  some  data  processing 
methods  such  as  bandwidth  normalizers  to  pre-process  the  data  and  improve  performance,  which 
were  specifically  designed  to  operate  with  dynamical  based  methods.  We  then  implemented 
some  novel  feature  space  discrimination  methods,  for  example  probabilistic  neural  networks,  to 
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additionally  improve  performance  by  providing  superior  methods  for  signal  class  separation. 
Finally,  we  tested  these  improvements  by  re-analyzing  a  data  set  of  dolphin  echolocation  chirps 
provided  by  SPA  WAR. 

Approach 

The  fundamental  technique  we  developed  was  a  method  whereby  the  unknown  coefficients  of  a 
dynamical  signal  model  could  be  reliably  and  robustly  estimated  from  observed  signals,  and  used 
to  generate  classification  features  for  Navy  detection  and  classification  applications.  This  method 
allows  the  unknown  model  coefficients  to  be  expressed  as  functions  of  generalized  signal 
correlations,  in  much  the  same  way  that  the  Yule- Walker  equations  are  used  for  AR  modeling 
methods.  This  method  has  the  great  advantage  of  typically  providing  closed-form  analytic 
expressions  for  the  unknown  coefficients,  for  general  classes  if  input  data.  This  has  allowed  us  to 
investigate  the  convergence  properties  and  signal  discrimination  in  a  rigorous  fashion.  We  have 
been  able  to  identify  correlation  functionals  which  are  explicitly  included  in  continuous 
dynamical  models  which  do  not  exist  in  standard  signal  models,  and  which  thereby  can  add  new 
classification  information.  We  have  used  this  formalism  to  generate  theoretical  performance 
estimates  for  idealized  signal  classes,  and  to  design  optimal  'dynamical  detectors'.  We  have  also 
applied  the  general  classification  ideas  to  a  few  real  data  sets  to  assess  the  methods  performance. 

Within  the  last  two-year  program  period,  we  desired  to  improve  the  classification  performance 
by  deriving  dynamical  models  which  were  specific  to  particular  data  sets.  We  pursued  two  main 
methods:  that  of  calculating  orthogonolized  model  expansions  using  a  Gram-Schmidt  technique, 
and  the  use  of  a  specially-formulated  genetic  algorithm  which  can  search  for  an  optimal  model  in 
an  automated  fashion.  Both  of  these  methods  greatly  improved  the  ability  to  model  nonlinear 
data  sets,  although  we  found  that  classification  performance  can  occasionally  depend  on  other 
issues  not  directly  related  to  the  data  model. 

We  also  attempted  to  improve  the  performance  of  the  dynamical  methods  by  adapting  known 
pre-  and  post-processing  methods  specifically  for  a  dynamical  paradigm.  For  pre-processing  of 
data,  we  developed  data  normalizes  which  allowed  us  to  filter  out  corrupting  signal  components, 
while  at  the  same  time  retaining  the  high  dimensionality  required  for  the  dynamical  classifiers. 

In  terms  of  post-processing,  we  investigate  several  methods  to  improve  discrimination  in  the 
feature  space  of  the  dynamical  classifiers,  since  the  dynamical  features  often  are  distributed  in  a 
non-Gaussian  fashion.  We  determined  that  probabilistic  neural  networks  provided  the  best 
discrimination  of  signal  classes  when  these  features  are  not  simply  connected. 

Finally  to  test  the  overall  performance  of  the  dynamical  classification  schemes,  we  analyzed  data 
sets  of  dolphin  acoustic  echolocation  data  provided  to  us  by  SPA  WAR.  During  the  previous 
program  period,  we  obtained  very  poor  performance  on  this  data  set,  however  the  improved 
techniques  developed  during  the  latter  program  period  provided  excellent  discrimination. 

S&T  Completed 
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During  the  first  program  period,  we  formulated  a  model  estimation  procedure  for  delay- 
differential  equations  into  a  rigorous  framework  for  classification  purposes.  We  calculated 
theoretical  feature  distributions  for  a  variety  of  idealized  signal  classes.  We  developed  a  pre¬ 
processor  to  determine  if  a  dynamical  model  is  useful  on  a  given  data  set,  by  evaluating  the 
dynamical  correlation  functions.  We  also  demonstrated  that  these  features  provide 
detection/classification  performance  beyond  that  achievable  by  standard  spectral  or  energy 
methods,  by  calculating  ROC  curves  using  a  generalized  Mahalanobis  distance  classifier.  We 
also  demonstrated  improved  detection  performance  on  two  real  data  sets  of  broadband,  low-SNR 
signals  from  sea  tests. 

In  the  most  recent  program  period,  we  achieved  several  important  results  which  we  believe  make 
the  dynamical  classification  methods  applicable  to  real-world  data  processing  applications. 
Firstly,  we  demonstrated  that  new  pre-processing  normalizes  are  capable  of  spectrally  removing 
most  types  of  signal  clutter  while  at  the  same  time  maintaining  high-dimensionality  of  the  signal, 
thereby  allowing  the  dynamical  models  to  accurately  represent  the  remaining  signal  with  respect 
to  a  background  noise  process. 

Secondly,  we  were  able  to  demonstrate  that  dynamical  signal  models  (i.e.  delay-differential 
equation  models)  could  be  calculated  which  were  specific  to  a  given  signal  class.  Gram-Schmidt 
orthogonalization  provides  one  such  method,  and  can  result  in  enormous  classification 
performance  increases  (up  to  15  dB),  however  this  technique  tends  to  be  “brittle”  with  regards  to 
variations  in  observed  data.  Model  selection  using  a  genetic  algorithm  was  found  to  provide  a 
much  more  robust  method,  however  overall  improvement  in  classification  performance  proves  to 
be  less  using  this  technique  (on  the  order  of  5-6  dB).  An  example  of  this  detection  performance 
improvement  is  given  in  Figure  1  below,  which  shows  a  ROC  curve  for  a  generic  signal  model 
(red),  and  for  a  signal-specific  model  calculated  with  the  genetic  algorithm  using  our  automated 
search  algorithm  (blue).  Both  were  calculated  on  the  identical  data  sets. 

Generic  Signal  Model 


dxldt-a.x,  +a,xT  +  a.xTxT 

1  Tj  z  r2  l  ^  t2 
Genetic  Algorithm  Derived  Model 

dxl  dt  =  a.xr  +a,x  + 

1  Tj  2  r2 

+  a,x2r  +  a4x3  +  d'Xl  xr 

O  T2  4  5  Tj  T2 
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Thirdly,  because  the  feature  distributions  for  dynamical  models  are  sometimes  distributed  in  a 
non-simply  connected  fashion,  we  adapted  new  feature  discrimination  methods  to  improve 
performance  during  post-processing.  The  most  powerful  technique  we  developed  utilized 
probabilistic  neural  networks,  which  provides  feature  partitioning  via  a  radial  basis  function 
expansion.  This  method  proved  able  to  discriminate  feature  distributions  from  different  signal 
classes  even  when  they  were  quite  non-Gaussian. 

Finally,  as  a  test  bed  for  the  new  techniques  developed  under  this  program  period,  we  analyzed  a 
data  set  of  dolphin  acoustic  echolocation  data  provided  by  SPA  WAR.  This  data  set  consisted  of 
trains  of  echolocation  pulses  which  were  categorized  according  to  target  geometry.  During  the 
previous  program  period,  we  were  not  able  to  effectively  discriminate  the  different  signal  classes 
using  the  more  basic  dynamical  classifiers.  However,  using  the  improvements  developed  during 
this  program  period,  we  were  able  to  generate  near-perfect  discrimination  of  these  different 
signal  classes.  Figure  2  shows  the  feature  distributions  and  classification  table  for  these  results. 


class  1 

class  2 

class  1  identified  as 

class  1  class  2 

class  2  identified  as 

class  1  class  2 

target  1 

target  2 

100% 

0% 

0% 

100  % 

target  1 

target  4 

100% 

0% 

0% 
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target  2 

target  4 

94% 

6% 

33% 
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target  1 
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0% 

0% 
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noise 

100% 

0% 

0% 

100  % 

target  4 

noise 

100  % 

0% 

0% 

100  % 
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This  research  provides  perhaps  the  first  method  to  embed  a  nonlinear  dynamical  processing 
technique  within  a  rigorous  statistical  discrimination  framework,  and  to  analytically  calculate  the 
theoretical  detection/classification  performance.  It  is  clear  from  several  examples  on  simulated 
and  real-world  data  sets  that  these  “dynamical  classifiers”  provide  enhanced  performance  over 
existing  methods  on  certain  classes  of  data.  Since  the  estimation  procedure  is  generally  fast  and 
robust,  adaptation  into  existing  Navy  processing  schemes  should  be  straightforward,  and  should 
produce  enhanced  detection/classification  performance  in  many  applications.  These  applications 
should  include  data  classes  which  are  particularly  difficult  for  conventional  energy  or  spectral 
classifiers  to  process,  for  example  quasi-broadband  signals  produced  by  low-dimensional 
dynamical  processes,  such  as  machine  resonances.  What  remains  is  to  provide  for  validated 
testing  in  a  real-world  setting  in  conjunction  with  Navy  researchers. 

Planned  S&T  Research  Efforts 

No  further  research  is  planned  at  this  time  due  to  termination  of  the  program. 

Technology  Transfer 

At  present,  we  are  continuing  to  interface  with  the  ONR  passive  sonar  program  to  attempt  to 
define  a  transition  path  for  the  dynamical  classifiers  to  the  6. 2/6. 3  arena.  This  will  primarily 
involve  an  assessment  on  real  world  data  sets,  which  is  being  coordinated  in  conjunction  with 
NUWC  researchers.  This  technology  has  also  drawn  considerable  interest  from  other  DoD 
organizations,  particularly  the  Army  acoustic  classification  program. 
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